Methods, systems, and media for certifying a playback device

ABSTRACT

Methods, systems, and media for certifying a playback device are provided. In some embodiments, methods for certifying a playback device are provided, the methods comprising: identifying a plurality of playback capabilities associated with the playback device; identifying at least one certification test based on the plurality of playback capabilities; receiving a plurality of media streams corresponding to the certification test from a server; causing a first media stream to be presented by the playback device; changing a streaming condition experienced by the playback device; causing a second media stream to be presented by the playback device based on the streaming condition; capturing, using a camera, a video signal representing video content presented by the playback device; and determining, using a hardware processor, whether the playback device has passed the certification test based at least in part on the video signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

The current application is a reissue of application Ser. No. 14/145,457,filed Dec. 31, 2013, which issued as U.S. Pat. No. 9,332,290.

TECHNICAL FIELDS

Methods, systems, and media fur certifying a playback device areprovided. More particularly, the disclosed subject matter relates tocertifying playback capabilities (e.g., adaptive streaming capabilities)of a playback device using a device including a camera and/or amicrophone.

BACKGROUND OF THE INVENTION

There are many conventional approaches to certifying a playback devicethat can decode and present audio content and/or video content. Forexample, a playback device can receive a video stream from a server andpresent the video stream on a display device. While the video stream isdisplayed on the display device, a user can determine whether the videostream is displayed with a desired video quality. However, manualcertification is time consuming and inaccurate. The result of thecertification heavily relies on the user's skills, lighting, and otherfactors which can be unpredictable.

Accordingly, new mechanisms for certifying playback devices aredesirable.

SUMMARY OF THE INVENTION

In view of the foregoing, systems, methods, and media for certifying aplayback device are provided. In some embodiments, methods forcertifying a playback device are provided, the methods comprising:identifying a plurality of playback capabilities associated with theplayback device; identifying at least one certification test based onthe plurality of playback capabilities; receiving a plurality of mediastreams corresponding to the certification test from a server; causing afirst media stream to be presented by the playback device; changing astreaming condition experienced by the playback device; causing a secondmedia stream to be presented by the playback device based on thestreaming condition; capturing, using a camera, a video signalrepresenting video content presented by the playback device; anddetermining, using a hardware processor, whether the playback device haspassed the certification test based at least in part on the videosignal.

In some embodiments, systems for certifying a playback device areprovided, the systems comprising: at least one hardware processor thatis configured to: identify a plurality of playback capabilitiesassociated with the playback device; identify at least one certificationtest based on the plurality of playback capabilities; receive aplurality of media streams corresponding to the certification test froma server; cause a first media stream to be presented by the playbackdevice; change a streaming condition experienced by the playback device;cause a second media stream to be presented by the playback device basedon the streaming condition; receive a video signal representing videocontent presented by the playback device; and determine whether theplayback device has passed the certification test based at least in parton the video signal; and a camera that is configured to capture thevideo signal.

In some embodiments, non-transitory computer-readable media containingcomputer-executable instructions that, when executed by a processingcircuitry, cause the processing circuitry to perform a method forcertifying a playback device are provided, the method comprising:identifying a plurality of playback capabilities associated with theplayback device; identifying at least one certification test based onthe plurality of playback capabilities; receiving a plurality of mediastreams corresponding to the certification test from a server; causing afirst media stream to be presented by the playback device; changing astreaming condition experienced by the playback device; causing a secondmedia stream to be presented by the playback device based on thestreaming condition; capturing a video signal representing video contentpresented by the playback device; and determining whether the playbackdevice has passed the certification test based at least in pats on thevideo signal.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the invention will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 shows a generalized block diagram of an example of anarchitecture of hardware that can be used to certify a playback devicein accordance with some embodiments of the disclosed subject matter;

FIG. 2 shows a flow chart of an example of a process for certifying aplayback device in accordance with some embodiments of the disclosedsubject matter;

FIG. 3 shows a flow chart of an example of a process for certifyingvideo and/or audio playback capabilities in accordance with someembodiments of the disclosed subject matter; and

FIG. 4 shows a flowchart of an example of a process for performing acertification test to certify adaptive bitrate streaming, capabilitiesof a playback device in accordance with some embodiments of thedisclosed subject matter.

DETAILED DESCRIPTION

Mechanisms (which can be systems, methods, media, etc.) for certifying aplayback device are provided.

In some embodiments, the mechanisms can certify various playbackcapabilities (e.g., adaptive streaming capabilities) of a playbackdevice using a device including a camera and/or a microphone, such as amobile phone, a tablet computer, a wearable computer, a gaming device,and/or any other suitable mobile device.

In some embodiments, the mechanisms can discover a playback device andidentify one or more playback capabilities associated with the playbackdevice which are to be certified. In some embodiments, the playbackcapabilities can include presenting video content and/or audio contentwith desired quality. In some embodiments, the playback capabilities caninclude one or more streaming functions that can be performed by theplayback device (e.g., adaptive streaming functions).

In some embodiments, the mechanisms can identify one or morecertification tests that can be used to certify the identified playbackcapabilities. The mechanisms can then receive test data from a serverand perform the certification tests on the playback device using thetest data.

In some embodiments, the mechanisms can transmit a test media streamincluding suitable video content and/or audio content to the playbackdevice and cause the video content and/or the audio content to bepresented by the playback device. In some embodiments, the mechanismscan capture a video signal representing the video content presented bythe playback device using a suitable camera. Additionally oralternatively, the mechanisms can capture an audio signal representingthe audio content presented by the playback device using a suitablemicrophone.

In some embodiments, the mechanisms can analyze the video signal and/orthe audio signal and determine whether the playback device has passedthe certification test based on the analysis. For example, themechanisms can determine whether the video content and the audio contentare presented in a synchronized manner based on the video signal and theaudio signal. In some embodiments, the mechanisms can determine that theplayback device has failed the certification test in response todetermining that the video content and the audio content are notpresented in a synchronized manner.

In some embodiments, the mechanisms can change a streaming conditionthat is experienced by the playback device (e.g., by performingbandwidth throttling). The mechanisms can then determine whether are-buffering event has occurred when the playback device switches to avideo stream encoded at a lower resolution. In some embodiments, themechanisms can determine that the playback device has failed thecertification test in response to detecting a re-buffering event.

Turning to FIG. 1, a generalized block diagram of an example 100 of anarchitecture of hardware that can be used to certify a playback devicein accordance with some embodiments is shown. As illustrated,architecture 100 can include a certification device 102, one or morecameras 104, one or more microphones 106, a playback device 108, adisplay device 110, a communication network 112, one or more servers114, and communication paths 116, 118, 120, 122, 124, 126, 128, and 130.

Certification device 102 can include any suitable circuitry that iscapable of capturing, processing, and/or analyzing video content andaudio content presented by a playback device, performing networkmanipulation tasks (e.g., bandwidth throttling, service redirection,and/or any other suitable network manipulation task), streaming mediastreams to a playback device at variable bits, communicating with aplayback device and a server, and/or performing any other suitablefunctions. Examples of certification device 102 can include a mobilephone, a tablet computer, a laptop computer, and/or any other suitabledevice.

In some embodiments, certifications device 102 can communicate withplayback device 108 through one or more communication channels (e.g., aWIFI connection, a BLUETOOTH connection, an ,Ethernet connection, and/orany other suitable communication channel). In some embodiments,certification device 102 can select a communication channel using alowest-common denominator communication method and perform one or morecertification tests on playback device 108 using the selectedcommunication channel.

In some embodiments, certification device 102 can be coupled to one ormore cameras 104 via a communication path 124. Camera(s) 104 can includeany suitable circuitry that is capable of capturing still images and/ormoving images, such as a charge-coupled device (CCD) sensor, acomplementary metal-oxide-semiconductor (CMOS) sensor, and/or any othersuitable circuitry. Examples of camera(s) 104 can include a digitalcamera, a camcorder, a video camera, a video recorder, a webcam, a PCcamera, and/or any other suitable camera.

In some embodiments, certification device 102 can be coupled to one ormore microphones 106 via a communication path 126. Microphone(s) 106 caninclude any suitable circuitry that is capable of receiving acousticinput from a user, capturing audio data from its surroundings,generating audio signals, processing audio signals, and/or performingother suitable functions.

In some embodiments, each of camera(s) 104 and microphone(s) 106 can beimplemented as one or more stand-alone devices or integrated withcertification device 102.

Playback device 108 can include any suitable circuitry that is capableof receiving, converting, processing, presenting, and/or transmittingmedia content, causing media content to be presented on display device110, and/or performing any other suitable functions. For example,playback device 108 can include a set-top box, a digital media receiver,a DVD player, a BLU-RAY player, a game console, a desktop computer, alaptop computer, a tablet computer, a mobile phone, a television device,and/or any other suitable device, and/or any suitable combination of thesame.

Display device 110 can include any suitable circuitry that is capable ofdecoding, encoding, and/or presenting media content, such as videocontent, audio content, and/or any other suitable content. For example,display device 110 can include a media center computer, a CRT display,an LCD, an LED display, a plasma display, a touch-screen display, asimulated touch screen, a television device, a tablet computer, a mobilephone, and/or any other suitable device. In some embodiments, displaydevice 104 can present a user with three-dimensional content.

In some embodiments, communication network 112 may be any one or morenetworks including the Internet, a mobile phone network, a mobile voicenetwork, a mobile data network (e.g., a 3G, 4G, or LTE network), a cabletelevision network, a satellite network, a public switched telephonenetwork, a local area network, a wide area network, a fiber-opticnetwork, any other suitable type of communications network, and/or anysuitable combination of these communications networks.

Server(s) 114 can include any suitable circuitry that is capable oftransmitting test data to certification device 102, receiving testresults from certification device 102, and/or performing any othersuitable functions.

In some embodiments, server(s) 114 can include one or more types ofcontent distribution equipment for distributing any suitable mediacontent, including television distribution facility equipment, cablesystem head-end equipment, satellite distribution facility equipment,programming source equipment (e.g., equipment of televisionbroadcasters, such as NBC, ABC, HBO, etc.), intermediate distributionfacility equipment, Internet provider equipment, on-demand media serverequipment, and/or any other suitable media content provider equipment.NBC is a trademark owned by the National Broadcasting Company, Inc., ABCis a trademark owned by the ABC, INC., and HBO is a trademark owned bythe Home Box Office, Inc.

Server(s) 114 may be operated by the originator of content (e.g., atelevision broadcaster, a Webcast provider, etc.) or may be operated bya party other than the originator of content (e.g., an on-demand contentprovider, an Internet provider of content of broadcast programs fordownloading, etc.).

Server(s) 114 may be operated by cable providers, satellite providers,on-demand providers, Internet providers, providers of over-the-topcontent, and/or any other suitable provider(s) of content.

Server(s) 114 may include a remote media server used to store differenttypes of content (including video content selected by a user) in alocation remote from any of the user equipment devices. For example,media Server(s) 114 can include one or more content delivery networks(CDN).

As referred to herein, the term “media content” or “content” should beunderstood to mean one or more electronically consumable media assets,such as television programs, pay-per-view programs, on-demand programs(e.g., as provided in video-on-demand (VOD) systems), Internet content(e.g., streaming content, downloadable content, Webcasts, etc.), movies,films, video clips, audio, audio books, and/or any other media ormultimedia and/or combination of the same. As referred to herein, theterm “multimedia” should be understood to mean media content thatutilizes at least two different content forms described above, forexample, text, audio, images, video, or interactivity content forms.Media content may be recorded, played, displayed or accessed by userequipment devices, but can also be part of a live performance. In someembodiments, media content can include over-the-top (OTT) content.Examples of OTT content providers include YOUTUBE, NETFLIX, and HULU,which provide audio and video via IP packets. Youtube is a trademarkowned by Google Inc., Netflix is a trademark owned by Netflix Inc., andHulu is a trademark owned by Hulu, LLC.

Media content can be provided from any suitable source in someembodiments. In some embodiments, media content can be electronicallydelivered to a user's location from a remote location. For example,media content, such as a Video-On-Demand movie, can be delivered to auser's home from a cable system server. As another example, mediacontent, such as a television program, can be delivered to a user's homefrom a streaming media provider over the Internet.

In some embodiments, each of certification device 102, camera(s) 104,microphone(s) 106, playback device 108, display device 110, andserver(s) 114 can be implemented in any suitable hardware. For example,each of certification device 102, camera(s) 104, microphone(s) 106,playback device 108, display device 110, and server(s) 114 can beimplemented in any of a general purpose device such as a computer or aspecial purpose device such as a client, a server, a mobile terminal(e.g., a mobile phone), and/or any other suitable device. Any of thesegeneral or special purpose devices can include any suitable componentssuch as a hardware processor (which can be a microprocessor, digitalsignal processor, a controller, and/or any other suitable hardwareprocessor). In some embodiments, each of certification device 102,camera(s) 104, microphone(s) 106, playback device 108, display device110, and server(s) 114 can include a suitable storage device, such asrandom-access memory, read-only memory, hard drives, optical drives,digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAYdisc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders(DVR, sometimes called a personal video recorder, or PVR), solid statedevices, quantum storage devices, gaming consoles, gaming media, or anyother suitable fixed or removable storage devices, and/or anycombination of the same.

In some embodiments, each of certification device 102, camera(s) 104,microphone(s) 106, playback device 108, display device 110, andserver(s) 114 can be implemented as a stand-alone device or integratedwith other components of architecture 100. For example, in someembodiments, one or both of camera(s) 104 and microphone(s) 106 can beintegrated with certification device 102. As another example, playbackdevice 108 can be integrated with display device 110 in someembodiments.

In some embodiments, playback device 108 can be connected tocertification device 102 and display device 110 through communicationspaths 128 and 130, respectively. In some embodiments, certificationdevice 102, playback device 108, display device 110, and server(s) 114can be connected to communication network 112 through communicationspaths 116, 118, 120, and 122, respectively.

Communications paths 116, 118, 120, 122, 124, 126, 128, and 130 mayseparately or together include one or more communications paths, suchas, a satellite path, a fiber-optic path, a cable path, a path thatsupports Internet communications (e.g., IPTV), free-space connections(e.g., for broadcast or other wireless signals), or any other suitablewired or wireless communications path or combination of such paths, insome embodiments.

Turning to FIG. 2, an example 200 of a process for certifying a playbackdevice in accordance with some embodiments of the disclosed subjectmatter is shown. In some embodiments, process 200 cart be implemented bya hardware processor of one or more components of architecture 100 ofFIG. 1, such as a certification device 102.

As illustrated, process 200 can begin by identifying one or moreplayback capabilities associated with a playback device that are to becertified at 202. Any suitable playback capabilities can be certified insome embodiments. For example, the playback capabilities can includepresenting video content with desired video quality. In a moreparticular example, video quality can be measured by one or morecharacteristics of video content presented by the playback device, suchas a video resolution (e.g., 480 p, 576 p, 720 p, 1080 p, 4 k, and/orany other suitable video resolution), a bit depth, a video format (e.g.,MPEG-4/ASP, AVI, DIVX, H.264, MKV, the High Efficiency Video Coding(HEVC), and/or any other suitable video format), a frame rate, a peaksignal-to-noise ratio, and/or any other suitable characteristic of videocontent.

As another example, the playback capabilities can include presentingaudio content with desired audio quality. In a more particular example,audio quality can be measured by one or more characteristics of audiocontent presented by the playback device, such as an audio frame rate,an audio format (e.g., WAV, WMA, MKV, MP3, and/or any other suitableaudio format), a bit depth, and/or any other suitable characteristic ofaudio content.

As yet another example, the playback capabilities can include one ormore streaming functions that can be performed by the playback device,such as adaptive bitrate streaming, trick play (e.g., fast-forward,rewind, resume playback across devices, and/or any other suitable trickplay function), and/or any other suitable streaming function.

In some embodiments, the playback capabilities to be certified can beidentified in any suitable manner. For example, the certification devicecan query the playback device to obtain the playback capabilities thatare to be certified. In a more particular example, the certificationdevice can scan a barcode (e.g., a two-dimensional barcode) displayed bythe playback device (on a screen of the playback device and/or a screenof a display device coupled to the playback device) and process thebarcode to obtain the playback capabilities associated with the playbackdevice.

As another example, the certification device can obtain an identifierassociated with the playback device (e.g., a product identifier, anapplication identifier, and/or any other suitable identifier associatedwith the certification device) and/or any other suitable informationabout the playback device (e.g., by querying the playback device). Thecertification device can then obtain information about playbackcapabilities associated with the playback device based on the identifierassociated with the certification device (e.g., by querying a databasethat stores such information).

At 204, process 200 can identify one or more certification tests to beperformed on the playback device based on the identified playbackcapabilities. Any suitable certification test can be used to certify oneor more of the identified playback capabilities. For example, acertification test can be performed to determine whether the playbackdevice is capable of presenting video content and/or audio content withthe video quality and/or the audio quality identified at 202. As anotherexample, a certification test can be performed to determine whether theplayback device is capable of performing one or more of the streamingfunctions identified at 202.

At 206, process 200 can receive test data from a server. The test datacan include any suitable information that can be used to perform theidentified certification tests on the playback device. For example, thetest data can include one or more test media streams to be processed andpresented by the playback device during the certification tests. In someembodiments, a test media stream can include any suitable media contentthat can be used to certify one or more playback capabilities, such asvideo content, audio content, captioning content, text, on-screendisplays, and/or any other suitable content. In some embodiments, themedia streams can include video content encoded at various resolutions,frame rates, bitrates, and/or any other suitable encoding parameters.

As another example, the test data can include license keys, streamconfiguration parameters, and/or any other suitable data that can beused to perform a certification test.

At 208, process 200 can perform a certification test on the playbackdevice using the test data. The certification test can be performed inany suitable manner. For example, process 200 can cause one or more testmedia streams to be presented by the playback device and capture a videosignal representing video content presented by the playback device andan audio signal representing audio content presented by the playbackdevice. Process 200 can then analyze the captured video signal and/orthe captured audio signal to determine whether the playback devicepossesses the identified playback capabilities. In a more particularexample, a certification can be performed using process 300 of FIG. 3and/or process 400 of FIG. 4 as described hereinbelow.

As another example, process 200 can verify whether the playback deviceis capable of negotiating a shared secret using a suitablekey-generation protocol, such as the Diffie-Hellman (DH) protocol. In amore particular example, process 200 can determine whether the playbackdevice is capable of generating a secure cipher key under the DHprotocol. Additionally or alternatively, process 200 can determinewhether the playback device is capable of encrypting and/or decryptingcommunications between the certification device and the playback deviceusing the secure cipher key.

As yet another example, process 200 can verify the integrity of theplayback device. In a more particular example, process 200 can receive acertification associated with the playback device (e.g., a SecureSockets Layer (SSL) certificate) and validate the certification using asuitable authentication scheme. In another more particular example,process 200 can receive a digital signature signed by the playbackdevice and verify the digital signature.

As still another example, process 200 can determine whether the playbackdevice is capable of implementing a security protocol, such as the SSL,the Transport Layer Security (TLS), and/or any other suitable securityprotocol. In a more particular example, process 200 can transmit to theplayback device an invalid certificate associated with a server anddetermine whether the playback is capable of detecting and rejecting theinvalid certificate. In another more particular example, process 200 cantransmit to the playback device a valid certificate associated with aserver and determine whether the playback device is capable ofvalidating the valid certificate in a suitable manner.

In some embodiments, process 200 can store and/or transmit the result ofthe certification test at 210. In some embodiments, the result of thecertification test can include whether the playback device has passedthe certification test and/or any other suitable information about theplayback device's performance during the certification test.

At 212, process 200 can determine whether one or more certificationtests are to be performed on the playback device. In some embodiments,process 200 cart loop back to 208 in response to determining that one ormore certification tests are to be performed on the playback device.

Alternatively, process 200 can transmit data about the results of thecertification tests to the server at 214.

Turning to FIG. 3, a flowchart of an example 300 of a process forcertifying video and/or audio playback capabilities in accordance withsome embodiments of the disclosed subject matter is shown. In someembodiments, process 300 can be implemented by a hardware processor ofone or more components of architecture 100 of FIG. 1, such as acertification device 102.

As illustrated, process 300 can begin by causing test media stream to bepresented by a playback device at 302. The test media stream can includeany suitable media content that can be used to verify one or more of thevideo and/or audio playback capabilities of the playback device. Forexample, the test media stream can include video content with one ormore given characteristics, such as a video resolution, a frame rate, avideo format, a bit depth, a bit rate, and/or any other suitablecharacteristic. As another example, the test media stream can includeaudio content with one or more given characteristics, such as an audioframe rate, a bit depth, an audio format, and/or any other suitablecharacteristic.

In some embodiments, the test media stream can be presented by theplayback device in any suitable manner. For example, the test mediastream can be presented on a display device that is coupled to and/orintegrated with the playback device.

At 304, process 300 can capture a video signal representing videocontent that is presented by the playback device. The video signal canbe captured in arty suitable manner. For example, the certificationdevice can take one or more pictures of the test media stream beingdisplayed on the display device. In a more particular example, thecertification device can control one or multiple cameras (e.g., built-incameras, external cameras, and/or any other suitable cameras) to takeone or more pictures of the screen of the display device or a suitableportion of the screen while the test media stream is being displayed onthe display device.

As another example, the certification device can capture one or morescreenshots of the test media stream being presented on the displaydevice. In a more particular example, while the test media stream isbeing displayed on the display device, the certification device cancapture a screenshot of the screen of the display device, an activewindow application presented on the screen, and/or any suitable portionof the screen. In some embodiments, the certification device canretrieve an image of the video content displayed on the display devicefrom a frame buffer in the display device. In another more particularexample, the certification device can record the screen output of thedisplay device while the test media stream is being displayed. Thecertification device can then generate a set of images of the displayedtest media stream (e.g., a screencast).

At 306, process 300 can capture an audio signal representing audiocontent presented by the playback device. The audio signal can becaptured in any suitable manner. For example, the certification devicecan activate a microphone that can capture audio data from itssurroundings and record the audio data while the test media stream ispresented by the playback device.

At 308, process 300 can analyze the video signal. For example, process300 can determine a set of presentation times of a set of video framesof the captured video signal. In a more particular example, apresentation time of a video frame can be determined based on atimestamp corresponding to the start of the presentation of the videoframe, a timestamp corresponding to the end of the presentation of thevideo frame, and/or any other suitable timestamp associated with thepresentation of the video frame.

As another example, process 300 can retrieve a reference video signalrepresenting the video content of the test media stream and identify oneor more video frames of the reference video signal that match one ormore video frames of the captured video signal. In a more particularexample, a video frame of the reference video signal can be regarded asa match to a video frame of the captured video signal and thus amatching frame when the difference between the two video frames does notexceed a predetermined threshold.

In some embodiments, upon identifying a set of matching video frames(e.g., a set of video frames of the reference video signal that match aset of video frames of the captured video signal), process 300 candetermine a set of presentation times corresponding to the set ofmatching video frames. In a more particular example, a presentation timeof a matching video frame can be determined based on a presentationtimestamp (PTS) associated with the matching video frame.

At 310, process 300 can analyze the captured audio signal. For example,the certification device can calculate the frame rate, frequencyresponse, bit rate, and/or any other suitable characteristic of thecaptured audio signal.

As another example, the certification device can identify one or moreaudio frames of the captured audio signal that correspond to one or morevideo frames of the captured video frame. The certification device canthen determine a set of presentation times of the identified audioframes. In some embodiments, an audio frame can be regarded ascorresponding to a video frame when the audio frame and the audio frameare scheduled to be presented in a synchronized manner.

At 312, process 300 can determine whether the playback device has passedthe certification test. This determination can be made by thecertification device in any suitable manner. For example, thecertification device can determine that the playback device has failedthe certification test in some embodiments in which the playback deviceis incapable of presenting video content and audio content in asynchronized manner. In a more particular example, the certificationdevice can compare a presentation time of a video frame of the capturedvideo signal and a presentation time of an audio frame in the capturedaudio signal that corresponds to the video frame. The certificationdevice can then determine that the playback device has failed thecertification test in some embodiments in which the difference betweenthe two presentation times exceeds a predetermined threshold.

As another example, the certification device can determine that theplayback device has failed the certification test in response todetermining that the playback device is incapable of presenting videocontent with desired video quality. In a more particular example, thecertification device can compare one or more video frames of capturedvideo signal and one or more matching frames of the reference videosignal. The certification device can then determine that the playbackdevice has failed the certification in some embodiments in which thedifference between the video frames of the captured video signal and thematching video frames exceeds a predetermined threshold.

Additionally or alternatively, the certification device can compare apresentation time of a video frame of the captured video signal and apresentation time of a matching video frame of the reference videosignal. The certification device can then determine that the playbackdevice has failed the certification test in some embodiments in whichthe difference between the two presentation times exceeds apredetermined threshold.

As yet another example, the certification device can determine that theplayback device has failed the certification test when the playbackdevice is incapable of presenting audio content with desired audioquality. In a more particular example, the certification device cancompare the captured audio signal and a reference audio signalrepresenting the audio content contained in the test media stream. Thecertification device can then determine that the playback device hasfailed the certification test in some embodiments in which thedifference between the captured audio signal and the reference audiosignal exceeds a predetermined threshold.

Turning to FIG. 4, a flowchart of an example 400 of a process forperforming a certification test to certify adaptive bitrate streamingcapabilities of a playback device is shown. In some embodiments, process400 can be implemented by a hardware processor of one or more componentsof system 100 of FIG. 1, such as a certification device 102 of FIG. 1.

As illustrated, process 400 can begin by receiving a set of test mediastreams including video content having different versions from a serverat 402. For example, the set of test media streams can include videocontent encoded at multiple bit rates, resolutions, frame rates, and/orany other suitable encoding parameters.

At 404, process 400 can redirect service requests sent by a playbackdevice to a certification device. This redirection can be performed inany suitable manner. For example, one or more requests to stream mediacontent from the server can be redirected to the certification device.

At 406, process 400 can transmit to a playback device a first test mediastream including video content having a first version. The first testmedia stream can include any suitable data that can be used to presentthe video content. For example, the first test media stream can includea set of video frames encoded at a given resolution, frame rate, bitrate, bit depth, and/or any other suitable encoding parameter. Asanother example, the first test media stream can include parameters thatcan be used to decode, process, and/or present the set of video frames,audio content to be presented with the set of video frames, and/or anyother suitable data.

At 408, process 400 can change a streaming condition experienced by theplayback device. Any suitable streaming condition can be changed in asuitable manner to verify one or more streaming functions that can beperformed by the playback device. For example, process 400 can changethe network bandwidth that can be utilized to transmit media streamsand/or messages between the certification device and the playbackdevice. In a more particular example, process 400 can constrain thenetwork bandwidth to a suitable bitrate using a suitable bandwidththrottling technique.

At 410, process 400 can receive a request for video content having asecond version from the playback device. The request can be received inany suitable manner. For example, the request can be directed to theserver from the playback device and can be redirected to thecertification device in some embodiments.

In some embodiments, the video content having the second version cancorrespond to a lower quality version of the video content having thefirst version. For example, the video content having the second versioncan include a set of frames encoded at a lower resolution, a lower bitrate, and/or any other suitable coding parameter.

At 412, process 400 can transmit to the playback device a second testmedia stream containing video content having the second version. In someembodiments, the second test media stream can include a set of videoframes encoded at a second resolution, a second frame rate, a second bitrate, a second bit depth, and/or any other suitable encoding parameter.As another example, the second test media stream can include parametersthat can be used to decode, process, and/or present the set of videotraffics, audio content to be presented with the set of video frames,and/or any other suitable data.

At 414, process 400 can capture a video signal representing videocontent being presented by the playback device. In some embodiments,block 414 can be performed in substantially the same manner as block 304of FIG. 3 as described above. For example, the video signal can becaptured by a suitable camera when the first media stream and the secondmedia stream are presented on a display (e.g., a display of the playbackdevice or a display of a display device coupled to the playback device)via the playback device.

At 416, process 400 can determine whether a re-buffering event hasoccurred during the playback of the first test media stream and thesecond test media stream. This determination can be made in any suitablemanner. For example, a re-buffering event can be regarded as havingoccurred when a time interval between the presentation of last videoframe of the first test media stream and the presentation of the firstvideo frame of the second test media stream exceeds a threshold. In someembodiments, the time interval can be calculated based on a presentationtime of the last video frame of the first test media stream and apresentation time of the first video frame of the second test mediastream.

As another example, a re-buffering event can be regarded as havingoccurred when the delay in the presentation of the first video frame ofthe second test stream exceeds a threshold. In some embodiments, thedelay in the presentation of the first video frame of the second teststream can be calculated based on the difference between a presentationtime of the first frame of the second test stream and a presentationtime of a matching frame of a reference video signal.

In some embodiments, process 400 can determine that the playback devicehas failed the certification test at 418 in response to determining thata re-buffering event occurred during the playback of the first teststream and/or the second test stream.

Alternatively, process 400 can determine whether one or more changes tostreaming conditions are to be made at 420. In some embodiments, process400 can loop back to 408 in response to determining that one or morechanges to streaming conditions are to be made.

Alternatively, process 400 can determine that the playback device haspassed the certification test at 422.

It should be noted that processes 200, 300, and 400 of FIGS. 2, 3, and 4can be performed concurrently in some embodiments. It should also benoted that the above steps of the flow diagrams of FIGS. 2-4 may beexecuted or performed in any order or sequence not limited to the orderand sequence shown and described in the figures. Furthermore, it shouldbe noted, some of the above steps of the flow diagrams of FIGS. 2-4 maybe executed or performed substantially simultaneously where appropriateor in parallel to reduce latency and processing times. And stillfurthermore, it should be noted, some of the above steps of the flowdiagrams of FIGS. 2-4 may be omitted.

In some embodiments, any suitable computer readable media can be usedfor storing instructions for performing the mechanisms and/or processesdescribed herein. For example, in some embodiments, computer readablemedia can be transitory or non-transitory. For example, non-transitorycomputer readable media can include media such as magnetic media (suchas hard disks, floppy disks, and/or any other suitable magnetic media),optical media (such as compact discs, digital video discs, Blu-raydiscs, and/or any other suitable optical media), semiconductor media(such as flash memory, electrically programmable read only memory(EPROM), electrically erasable programmable read only memory (EEPROM),and/or any other suitable semiconductor media), any suitable media thatis not fleeting or devoid of any semblance of permanence duringtransmission, and/or any suitable tangible media. As another example,transitory computer readable media can include signals on networks, inwires, conductors, optical fibers, circuits, any suitable media that isfleeting and devoid of any semblance of permanence during transmission,and/or any suitable intangible media.

The provision of the examples described herein (as well as clausesphrased as “such as,” “e.g.,” “including,” and the like) should not beinterpreted as limiting the claimed subject matter to the specificexamples; rather, the examples are intended to illustrate only some ofmany possible aspects.

Accordingly, methods, systems, and media for certifying a playbackdevice are provided.

The above described embodiments of the present disclosure are presentedfor purposes of illustration and not of limitation, and the presentdisclosure is limited only by the claims which follow.

What is claimed is:
 1. A method for certifying capabilities of aplayback device using a certification device, the method comprising:displaying a test image on a display coupled to a playback device;capturing the test image displayed on the display coupled to theplayback device using a camera coupled to a certification device;processing the test image using the certification device to identify aplurality of capabilities associated with the playback device;identifying a particular set of certification tests based on theplurality of capabilities associated with the playback device using thecertification device, wherein the particular set of certification testscomprises: an interactivitya playback capabilities certification test, ashared secret certification test, a security protocol certificationtest, and a video and audio synchronization test; directing the playbackdevice to perform a series of operations according to the interactivityplayback capabilities certification test, the shared secretcertification test, the security protocol certification test, and thevideo and audio synchronization test using the playback device;capturing information produced by the playback device according to theseries of operations using the certification device; and determining,using a hardware processor, whether the playback device has passed thecertification tests according to captured information using thecertification device.
 2. The method of claim 1, wherein directing theplayback device to perform a series of operations according to theinteractivity playback capabilities certification test further comprisesdirecting the playback device to perform the following operations:receiving a plurality of media streams corresponding to a certificationtest from a server; causing a first media stream to be presented by theplayback device; changing a streaming condition experienced by theplayback device; causing a second media stream to be presented by theplayback device based on the streaming condition, wherein the firstmedia stream comprises a first plurality of video frames encoded at afirst resolution and wherein the second media stream comprises a secondplurality of video frames encoded at a second resolution.
 3. The methodof claim 2, wherein the first resolution is greater than the secondresolution.
 4. The method of claim 3, further comprising: determiningwhether a re-buffering event has occurred based at least in part on thevideo signal; and determining that the playback device has failed thecertification test in response to determining that a re-buffering eventhas occurred.
 5. The method of claim 1, further comprising performingbandwidth throttling.
 6. The method of claim 1, wherein directing theplayback device to perform a series of operations according to theinteractivity playback capabilities certification test further comprisesdirecting the playback device to perform the following operations:receiving a first test stream of interactivity content from a server onthe playback device; playing back the first test stream of interactivitycontent using the playback device, wherein a video component of thefirst test stream is displayed on the display device in an active windowapplication; receiving interactivity input to the first test stream ofinteractivity content on the playback device; capturing a screenshot ofthe active window application presented on the display device using acamera coupled to the certification device; and verifying that thescreenshot matches expected interactivity playback capabilitiesaccording to the interactivity playback capabilities certification testusing the certification device.
 7. The method of claim 1, whereindirecting the playback device to perform a series of operationsaccording to the video and audio synchronization test comprisesdirecting the playback device to perform the following operations:receiving a first test stream of media content corresponding to theparticular certification test from a server using the playback device;playing back the first test stream of media content using the playbackdevice, wherein a video component of the first test stream is displayedon the display device; capturing, using a camera coupled to thecertification device, a video signal from the display of the first teststream during playback; capturing, using a microphone coupled to thecertification device, an audio signal from the first test stream duringplayback; determining a presentation time of a video frame of the videosignal using the certification device; determining a presentation timeof an audio frame of the audio signal using the certification device;and when the presentation time of the video frame and the presentationtime of the audio frame differ beyond a predetermined threshold,determining that the playback device has failed the video and audiosynchronization test using the certification device.
 8. The method ofclaim 1 wherein the test image is a barcode identifying the playbackdevice, the certification device is a mobile device, and the playbackdevice is a television.
 9. The method of claim 1, further comprisingtransmitting a result of a certification test to the server.
 10. Themethod of claim 1, wherein directing the playback device to perform aseries of operations according to the shared secret certification testfurther comprises directing the playback device to perform the followingoperations: directing the playback device to generate a secure cipherkey according to a key-generation protocol using the certificationdevice; and verifying, using the certification device, that the playbackdevice successfully applied the secure cipher key to a testcommunication between the certification device and the playback device.11. The method of claim 1, wherein directing the playback device toperform a series of operations according to the security protocolcertification test further comprises directing the playback device toperform the following operations: transmitting an invalid certificateassociated with a server to the playback device; and verifying that theplayback device successfully rejects the invalid certificate using thecertification device.
 12. A system for certifying capabilities of aplayback device, the system comprising: a playback device; acertification device; and at least one hardware processor that isconfigured to: display a test image on a display coupled to the playbackdevice; capture the test image displayed on the display coupled to theplayback device using a camera coupled to the certification device;processing the test image using the certification device to identify aplurality of capabilities associated with the playback device; identifya particular set of certification tests based on the plurality ofcapabilities associated with the playback device using the certificationdevice, wherein the particular set of certification tests comprises: aninteractivitya playback capabilities certification test, a shared secretcertification test, a security protocol certification test, and a videoand audio synchronization test; direct the playback device to perform aseries of operations according to the interactivity playbackcapabilities certification test, the shared secret certification test,the security protocol certification test, and the video and audiosynchronization test using the playback device; capture informationproduced by the playback device according to the series of operationsusing the certification device; and determine whether the playbackdevice has passed the certification tests according to capturedinformation using the certification device.
 13. The system of claim 12,wherein directing the playback device to perform a series of operationsaccording to the interactivity playback capabilities certification testfurther comprises directing the playback device to perform the followingoperations: receive a plurality of media streams corresponding to acertification test from a server; cause a first media stream to bepresented by the playback device; change a streaming conditionexperienced by the playback device; causing a second media stream to bepresented by the playback device based on the streaming condition,wherein the first media stream comprises a first plurality of videoframes encoded at a first resolution and wherein the second media streamcomprises a second plurality of video frames encoded at a secondresolution.
 14. The system of claim 13, wherein the first resolution isgreater than the second resolution.
 15. The system of claim 14, whereinthe hardware processor is further configured to: determine whether are-buffering event has occurred based at least in part on the videosignal; and determine that the playback device has failed thecertification test in response to determining that a re-buffering eventhas occurred.
 16. The system of claim 12, wherein the hardware processoris further configured to perform bandwidth throttling,.
 17. The systemof claim 12, wherein directing the playback device to perform a seriesof operations according to the interactivity playback capabilitiescertification test further comprises directing the playback device toperform the following operations: receive a first test stream ofinteractivity content from a server on the playback device; playback thefirst test stream of interactivity content using the playback device,wherein a video component of the first test stream is displayed on thedisplay device in an active window application; receive interactivityinput to the first test stream of interactivity content on the playbackdevice; capture a screenshot of the active window application presentedon the display device using a camera coupled to the certificationdevice; and verify that the screenshot matches expected interactivitycapabilities according to the interactivity playback capabilitiescertification test using the certification device.
 18. The system ofclaim 12, wherein screenshot matches expected interactivity capabilitiesaccording to the interactivity certification test using thecertification device directing the playback device to perform a seriesof operations according to the playback capabilities certification testfurther comprises directing the playback device to perform the followingoperations: receive a first test stream of media content correspondingto the particular certification test from a server using the playbackdevice; playback the first test stream of media content using theplayback device, wherein a video component of the first test stream isdisplayed on the display device; capture, using a camera coupled to thecertification device, a video signal from the display of the first teststream during playback; capture, using a microphone coupled to thecertification device, an audio signal from the first test stream duringplayback; determine a presentation time of a video frame of the videosignal using the certification device; determine a presentation time ofan audio frame of the audio signal using the certification device; andwhen the presentation time of the video frame and the presentation timeof the audio frame differ beyond a predetermined threshold, determinethat the playback device has failed the video and audio synchronizationtest using the certification device.
 19. The system of claim 12, whereinthe test image is a barcode identifying the playback device, thecertification device is a mobile device, and the playback device is atelevision.
 20. The system of claim 12, wherein the hardware processoris further configured to transmit a result of a certification test tothe server.
 21. The system of claim 12, wherein directing the playbackdevice to perform a series of operations according to the shared secretcertification test further comprises directing the playback device toperform the following operations: direct the playback device to generatea secure cipher key according to a key-generation protocol using thecertification device; and verify, using the certification device, thatthe playback device successfully applied the secure cipher key to a testcommunication between the certification device and the playback device.22. The system of claim 12, wherein directing the playback device toperform a series of operations according to the interactivity securityprotocol certification test further comprises directing the playbackdevice to perform the following operations: transmit an invalidcertificate associated with a server to the playback device; and verifythat the playback device successfully rejects the invalid certificateusing the certification device.
 23. A non-transitory computer-readablemedium containing computer-executable instructions that, when executedby a processing circuitry, cause the processing circuitry to perform amethod for certifying capabilities of a playback device, the methodcomprising: displaying a test image on a display coupled to a playbackdevice; capturing the test image displayed on the display coupled to theplayback device using a camera coupled to a certification device;processing the test image using the certification device to identify aplurality of capabilities associated with the playback device;identifying a particular set of certification tests based on theplurality of capabilities associated with the playback device using thecertification device, wherein the particular set of certification testscomprises: an interactivitya playback capabilities certification test, ashared secret certification test, a security protocol certificationtest, and a video and audio synchronization test; directing the playbackdevice to perform a series of operations according to the interactivityplayback capabilities certification test, the shared secretcertification test, the security protocol certification test, and thevideo and audio synchronization test using the playback device;capturing information produced by the playback device according to theseries of operations using the certification device; and determiningwhether the playback device has passed the certification tests accordingto captured information using the certification device.
 24. Thenon-transitory computer-readable medium of claim 23, wherein directingthe playback device to perform a series of operations according to theinteractivity playback capabilities certification test further comprisesdirecting the playback device to perform the following operations:receiving a plurality of media streams corresponding to a certificationtest from a server; causing a first media stream to be presented by theplayback device; changing a streaming condition experienced by theplayback device; causing a second media stream to be presented by theplayback device based on the streaming condition, wherein the firstmedia stream comprises a first plurality of video frames encoded at afirst resolution and wherein the second media stream comprises a secondplurality of video frames encoded at a second resolution.
 25. Thenon-transitory computer-readable medium of claim 24, wherein the firstresolution is greater than the second resolution.
 26. The Eon-transitorycomputer-readable medium of claim 25, wherein the method furthercomprises: determining whether a re-buffering event has occurred basedat least in part on the video signal; and determining that the playbackdevice has failed the certification test in response to determining thata re-buffering event has occurred.
 27. The non-transitorycomputer-readable medium of claim 23, wherein the method furthercomprises performing bandwidth throttling.
 28. The non-transitorycomputer-readable medium of claim 23, wherein directing the playbackdevice to perform a series of operations according to the interactivityplayback capabilities certification test further comprises directing theplayback device to perform the following operations: receiving a firsttest stream of interactivity content from a server on the playbackdevice; playing back the first test stream of interactivity contentusing the playback device, wherein a video component of the first teststream is displayed on the display device in an active windowapplication; receiving interactivity input to the first test stream ofinteractivity content on the playback device; capturing a screenshot ofthe active window application presented on the display device using acamera coupled to the certification device; and verifying that thescreenshot matches expected interactivity capabilities according to theinteractivity playback capabilities certification test using thecertification device.
 29. The non-transitory computer-readable medium ofclaim 23, wherein directing the playback device to perform a series ofoperations according to the video and audio synchronization testcomprises directing the playback device to perform the followingoperations: receiving a first test stream of media content correspondingto the particular certification test from a server using the playbackdevice; playing back the first test stream of media content using theplayback device, wherein a video component of the first test stream isdisplayed on the display device; capturing, using a camera coupled tothe certification device, a video signal from the display of the firsttest stream during playback; capturing, using a microphone coupled tothe certification device, an audio signal from the first test streamduring playback; determining a presentation time of a video frame of thevideo signal using the certification device; determining a presentationtime of an audio frame of the audio signal using the certificationdevice; and when the presentation time of the video frame and thepresentation time of the audio frame differ beyond a predeterminedthreshold, determining that the playback device has failed the video andaudio synchronization test using the certification device.
 30. Thenon-transitory computer-readable medium of claim 23, wherein the testimage is a barcode identifying the playback device, the certificationdevice is a mobile device, and the playback device is a television. 31.The non-transitory computer-readable medium of claim 23, wherein themethod further comprises transmitting a result of a certification testto the server.
 32. The non-transitory computer-readable medium of claim23, wherein directing the playback device to perform a series ofoperations according to the shared secret certification test furthercomprises directing the playback device to perform the followingoperations: directing the playback device to generate a secure cipherkey according to a key-generation protocol using the certificationdevice; and verifying, using the certification device, that the playbackdevice successfully applied the secure cipher key to a testcommunication between the certification device and the playback device.33. The non-transitory computer-readable medium of claim 23, whereindirecting the playback device to perform a series of operationsaccording to the security protocol certification test further comprisesdirecting the playback device to perform the following operations:transmitting an invalid certificate associated with a server to theplayback device; and verifying that the playback device successfullyrejects the invalid certificate.